1. Field of the Invention
This invention relates to the field of optical instrument recognition, and more particularly to a system and method for automatically interpreting and analyzing gauges, readouts, and the position and state of user controls in an environment with highly dynamic lighting conditions.
2. Description of the Related Art
The recording and automated analysis of image data is well known in the prior art. For example, optical character recognition, or OCR, is the process of analyzing an image of a document and converting the printed text found therein into machine-editable text. OCR programs are readily available and often distributed for free with computer scanners and word editing programs. OCR is a relatively simple task for modern software systems, as documents are typically presented with known lighting conditions (that is, an image of dark text on a light background, captured with the consistent, bright exposure light of a document scanning system) using predetermined character sets (that is, known and readily-available character fonts).
Systems attempting to recognize handwritten text have the added challenge of handling the variations in personal handwriting styles from one person to the next. Still, these systems often require that the writers print the text instead of using cursive and that they follow certain guidelines when creating their printed characters. Even in these systems, where the individual style variations must be accounted for, the lighting conditions used to capture the text images are well-controlled and consistent.
Another example of automated image analysis is facial recognition. A facial recognition system is a computer application for automatically identifying a person from a digital image of the person's face. Facial recognition programs are useful in security scenarios, such as analyzing passengers boarding an aircraft in an attempt to identify known terrorists. A typical facial recognition program works by comparing selected facial features from the image, such as the distance between the person's eyes or the length of the nose, against a facial feature database. As with optical character recognition, facial recognition works best in controlled lighting conditions when the subject matter (that is, the face) is in a known orientation relative to the image.
It is also common to use video cameras in the cockpit of an aircraft or cab of a land-based vehicle as a means of gathering data. In the event of a crash or near-miss, the recorded video can be post-processed (that is, processed by experts and systems off-board the vehicle, after the image data has been downloaded to an external system) to determine what conditions were present in the vehicle during the incident. Storing the video data on board the vehicle requires a large amount of storage space. Because of this, mechanisms are often used to limit the amount of storage required on board the vehicle, such as only storing the most recent video data (for example, only storing the most recent 10 minutes of data, and overwriting anything older than this.)
The ambient lighting conditions of a vehicle cab or aircraft cockpit are highly dynamic, and vary based on the time of day, the angle of the vehicle in relation to the sun, and on the presence of other external sources of illumination. One portion of an instrument panel may be concealed in shadow, while another portion is bathed in direct sunlight. The dividing line between dark and light constantly changes as the vehicle maneuvers and changes position in relation to the sun. Commercially available camera systems for use in aircraft cockpits do not perform well in these conditions, and provide low-quality images. These limitations make the task of post-processing the image data to clearly identify details within the images difficult if not impossible.
A single clear image of an aircraft cockpit, however, would contain a wealth of information about the ongoing flight. An image of a cockpit would capture a snapshot of the current state of each of the flight instruments, the position of the pilot and copilot, and the presence of any unusual conditions (such as smoke) for any given moment in time. If automatic image analysis of this data could be consistently performed in real time, while the flight is in progress, this visual information could be interpreted and stored as numeric data and/or communicated to the pilot and/or other onboard systems. Further, if this image data could be captured by a self-contained camera module with built-in processing capabilities, the ability to process and analyze cockpit image data could be added to any vehicle, regardless if that vehicle had its own onboard computer or sensing systems. This stand-alone camera module could capture the image data while the flight or trip was in progress, analyze the image data and convert it to numeric data, and then compare that numeric data to pre-existing data, such as a flight plan or terrain model, already contained in the camera module.
What is needed in the art is an imaging system which can, in real time, capture high quality images of an aircraft cockpit or vehicle cab or portions thereof, compensate for the dynamic lighting conditions that can be present across even the area covered by a single gauge, analyze the image data and translate it into numeric data, and provide information and/or advisories to the pilots and other onboard systems. This system should also incorporate other information and capabilities such that it is aware of its own position and orientation in three-dimensional space and such that it can operate as a stand-alone unit, without the need to be tied into other onboard vehicle systems.